An accelerated Levenberg-Marquardt algorithm for feedforward network
نویسنده
چکیده
This paper proposes a new Levenberg-Marquardt algorithm that is accelerated by adjusting a Jacobian matrix and a quasi-Hessian matrix. The proposed method partitions the Jacobian matrix into block matrices and employs the inverse of a partitioned matrix to find the inverse of the quasi-Hessian matrix. Our method can avoid expensive operations and save memory in calculating the inverse of the quasi-Hessian matrix. It can shorten the training time for fast convergence. In our results tested in a large application, we were able to save about 20% of the training time than other algorithms.
منابع مشابه
An Algorithm for Fast Convergence in Training Neural Networks
In this work, two modifications on Levenberg-Marquardt algorithm for feedforward neural networks are studied. One modification is made on performance index, while the other one is on calculating gradient information. The modified algorithm gives a better convergence rate compared to the standard Levenberg-Marquard (LM) method and is less computationally intensive and requires less memory. The p...
متن کاملNNC Student Summer Research Support Program Report
This report presents the studies carried out on two modifications suggested in the literature for Levenberg-Marquardt algorithm. The modifications are applicable to feed-forward neural networks. One modification [18], made on performance index, reduces computational complexity of the Levenberg-Marquardt algorithm, while the other one [17], made on calculation of the gradient information, improv...
متن کاملCalibration of an Inertial Accelerometer using Trained Neural Network by Levenberg-Marquardt Algorithm for Vehicle Navigation
The designing of advanced driver assistance systems and autonomous vehicles needs measurement of dynamical variations of vehicle, such as acceleration, velocity and yaw rate. Designed adaptive controllers to control lateral and longitudinal vehicle dynamics are based on the measured variables. Inertial MEMS-based sensors have some benefits including low price and low consumption that make them ...
متن کاملAn Efficient Optimization Method for Extreme Learning Machine Using Artificial Bee Colony
Traditional learning algorithms with gradient descent based technique, such as back-propagation (BP) and its variant Levenberg-Marquardt (LM) have been widely used in the training of multilayer feedforward neural networks. The gradient descent based algorithm may converge usually slower than required time in training, since many iterative learning step are needed by such learning algorithm, and...
متن کاملApplying Bacterial Memetic Algorithm for Training Feedforward and Fuzzy Flip-Flop based Neural Networks
In our previous work we proposed some extensions of the Levenberg-Marquardt algorithm; the Bacterial Memetic Algorithm and the Bacterial Memetic Algorithm with Modified Operator Execution Order for fuzzy rule base extraction from inputoutput data. Furthermore, we have investigated fuzzy flip-flop based feedforward neural networks. In this paper we introduce the adaptation of the Bacterial Memet...
متن کامل